Conventionally, in an imaging apparatus having an imaging element (an image sensor such as CCD sensor, CMOS sensor, or the like), when the image sensor samples an image formed by a photographing lens system, aliasing noise such as false color, moiré fringes, or the like is generated.
In general, ordinary noise can be suppressed to some extent by taking statistical properties into account. However, aliasing noise is often globally generated on an image depending on the object, and it is difficult to apply an ordinary noise measure.
As one of methods of suppressing this aliasing noise, a method of estimating a pixel value which should be present between pixels by interpolation processing, and reconstructing an image before sampling is known. With this method, since the suppression amount of aliasing noise is influenced by the interpolation processing, it is important to select appropriate interpolation processing.
As a conventional interpolation processing method used in an imaging apparatus (imaging system), a method described in, e.g., T. Sakamoto et al., IEEE Trans. on Cons. Elec., 44, 1342-1352 (1998), and J. Mukherjee, Patt. Rec. Lett. 22, 339-351 (2001) are known. These interpolation processing methods used in the imaging system are used to reconstruct a completely captured image having full colors from an image sampled for respective colors by a color filter array. In addition, as methods based on a convolution operation, a nearest neighbor method, bilinear method, cubic convolution method, and the like are known.
As interpolation processing methods using object information captured in advance, a method of estimating a color strength value by assuming that a strength ratio of R, G and B, and G in a local region is constant, and a method of estimating a color after an object shape is divided into an edge portion and the like have been proposed.
Furthermore, an interpolation processing method that performs image reconstruction with high precision by applying an advanced image reconstruction technique using a generalized inverse matrix has been proposed. U.S. Pat. Nos. 5,294,976 and 5,450,128 have proposed an interpolation method which reconstructs an image using a sensitivity function that expresses a mapping from an original image to an observation image as a matrix.
Conventionally, it is possible to suppress aliasing noise by reconstructing an image before sampling by an image sensor. However, in order to suppress this aliasing noise, a high interpolation precision level is required. However, the nearest neighbor method based on the convolution operation, the method using color information or edge information of an object, and the like have low interpolation precision since they execute interpolation processing using only pixel values very near a pixel to be interpolated, thus lowering the suppression effect of aliasing noise.
Also, when only a mapping from an original image to an observation image is considered like in the interpolation reconstruction method of U.S. Pat. Nos. 5,294,976 and 5,450,128, aliasing noise cannot be controlled since it is generated depending on an object.
It is an object of the present invention to provide an interpolation processing method, an interpolation processing program, and an imaging apparatus, which can suitably suppress aliasing noise.